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Preface  &  Acknowledgements 


Welcome  to  our  Tenth  Annual  Acquisition  Research  Symposium!  We  regret  that  this 
year  it  will  be  a  “paper  only”  event.  The  double  whammy  of  sequestration  and  a  continuing 
resolution,  with  the  attendant  restrictions  on  travel  and  conferences,  created  too  much 
uncertainty  to  properly  stage  the  event.  We  will  miss  the  dialogue  with  our  acquisition 
colleagues  and  the  opportunity  for  all  our  researchers  to  present  their  work.  However,  we 
intend  to  simulate  the  symposium  as  best  we  can,  and  these  Proceedings  present  an 
opportunity  for  the  papers  to  be  published  just  as  if  they  had  been  delivered.  In  any  case,  we 
will  have  a  rich  store  of  papers  to  draw  from  for  next  year’s  event  scheduled  for  May  14-15, 
2014! 


Despite  these  temporary  setbacks,  our  Acquisition  Research  Program  (ARP)  here  at 
the  Naval  Postgraduate  School  (NPS)  continues  at  a  normal  pace.  Since  the  ARP’s 
founding  in  2003,  over  1,200  original  research  reports  have  been  added  to  the  acquisition 
body  of  knowledge.  We  continue  to  add  to  that  library,  located  online  at 
www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  70  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  encourage  your  future  participation. 

Unfortunately,  what  will  be  missing  this  year  is  the  active  participation  and 
networking  that  has  been  the  hallmark  of  previous  symposia.  By  purposely  limiting 
attendance  to  350  people,  we  encourage  just  that.  This  forum  remains  unique  in  its  effort  to 
bring  scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  It  provides  the  opportunity  to  interact  with  many  top  DoD 
acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both  in  the  formal 
panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals,  breaks,  and  the 
day-ending  socials.  Many  of  our  researchers  use  these  occasions  to  establish  new  teaming 
arrangements  for  future  research  work.  Despite  the  fact  that  we  will  not  be  gathered 
together  to  reap  the  above-listed  benefits,  the  ARP  will  endeavor  to  stimulate  this  dialogue 
through  various  means  throughout  the  year  as  we  interact  with  our  researchers  and  DoD 
officials. 

Affordability  remains  a  major  focus  in  the  DoD  acquisition  world  and  will  no  doubt  get 
even  more  attention  as  the  sequestration  outcomes  unfold.  It  is  a  central  tenet  of  the  DoD’s 
Better  Buying  Power  initiatives,  which  continue  to  evolve  as  the  DoD  finds  which  of  them 
work  and  which  do  not.  This  suggests  that  research  with  a  focus  on  affordability  will  be  of 
great  interest  to  the  DoD  leadership  in  the  year  to  come.  Whether  you’re  a  practitioner  or 
scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 
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Abstract 

Initiatives  to  reduce  ship  maintenance  costs  have  not  yet  realized  the  normal  cost-reduction 
learning  curve  improvements.  One  explanation  is  the  lack  of  recommended  technologies. 
Damen,  a  Dutch  shipbuilding  and  service  firm,  has  incorporated  similar  technologies  and  is 
developing  others  to  improve  its  operations.  This  research  collected  data  on  Dutch  ship 
maintenance  operations  and  used  it  to  build  three  types  of  computer  simulation  models  of 
ship  maintenance  and  technology  adoption.  Results  were  compared  with  previously 
developed  modeling  results  of  U.S.  Navy  ship  maintenance  and  technology  adoption. 
Adopting  3D  PDF  alone  improves  ROI  significantly  more  than  adopting  a  logistics  package 
alone,  and  adding  both  technologies  improves  ROI  more  than  adding  either  technology 
alone.  Adoption  of  the  technologies  would  provide  cost  benefits  far  in  excess  of  not  using  the 
technologies,  and  there  were  marginal  benefits  in  sequentially  implementing  the  technologies 
over  immediately  implementing  them.  Potential  benefits  of  using  the  technologies  are  very 
high  in  both  cases.  Implications  for  acquisition  practice  include  the  need  for  careful  analysis 
and  selection  from  among  a  variety  of  available  information  technologies  and  the 
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recommendation  for  a  phased  development  and  implementation  approach  to  manage 

uncertainty. 

Introduction 

The  current  cost-constrained  environment  within  the  federal  government  and  the 
DoD  requires  a  defensible  approach  to  cost  reductions  without  compromising  the  capability 
of  core  defense  processes  and  platforms.  Due  to  this  environment,  defense  leaders  today 
must  maintain  and  modernize  the  U.S.  armed  forces  to  retain  technological  superiority  while 
simultaneously  balancing  defense  budget  cost  constraints  and  extensive  military  operational 
commitments.  At  the  same  time,  defense  leaders  must  navigate  a  complex  information 
technology  (IT)  acquisition  process.  Maintenance  programs  play  a  critical  role  in  meeting 
these  DoD  objectives.  One  such  core  process  that  is  central  to  U.S.  naval  operations  is  the 
ship  maintenance  process.  This  process  alone  accounts  for  billions  of  dollars  in  the  U.S. 
Navy’s  annual  budget.  There  have  been  a  series  of  initiatives  designed  to  reduce  the  cost  of 
this  core  process,  including  ship  maintenance.  SHIPMAIN,  and  its  derivatives,  was  one  of 
the  initiatives  designed  to  improve  ship  maintenance  performance  within  the  Navy  by 
standardizing  processes  in  order  to  take  advantage  of  learning  curve  cost  savings. 

However,  these  process  improvement  initiatives  have  not  yet  realized  the  normal 
cost-reduction  learning  curve  improvements  for  common  maintenance  items  for  a  series  of 
common  platform  ships.  One  explanation  is  that  the  initial  instantiation  of  SHIPMAIN  did  not 
include  two  recommended  technologies,  three-dimensional  laser  scanning  technology  (3D 
LST)  and  collaborative  product  life-cycle  management  (CPLM),  that  were  deemed 
necessary  by  the  creator  of  SHIPMAIN  for  ensuring  the  success  of  the  new  standardized 
approach  (i.e.,  normal  learning  curve  cost  savings).  Previous  research  (Ford,  Housel,  & 

Mun,  2011)  indicates  that  adding  these  technologies  may  help  SHIPMAIN,  or  its  derivatives, 
to  capture  the  potential  savings.  But  the  technologies  have  not  been  implemented  to  date  in 
the  ship  maintenance  processes. 

However,  Damen,  a  large  shipbuilding  and  service  firm  has  incorporated  similar 
technologies  and  is  developing  others  to  improve  its  operations.  In  addition,  the  Royal  Dutch 
Navy  (RDN)  performs  all  of  its  own  ship  maintenance  in  a  single  yard  and  operation.  In  the 
current  study,  the  potential  benefits  of  similar  technologies  are  extrapolated  and  compared 
with  similar  projections  for  U.S.  Navy  ship  maintenance  processes.  These  organizations 
provide  a  source  of  relatively  reliable  data  on  operations  that  are  comparable  to  those 
performed  by  the  U.S.  Navy. 

Problem  Description 

Previous  research  on  the  potential  use  of  3D  LST  and  CPLM  technology  in  U.S. 

Navy  ship  maintenance  (e.g.,  Komoroski,  2005;  Ford,  Housel,  &  Mun,  2011)  estimated  the 
impacts  on  processes  due  to  technology  adoption.  Changes  such  as  reengineering  ship 
maintenance  processes,  the  sizes  of  reductions  in  cycle  times,  and  workforce  requirements 
are  examples  of  model  portions  that  required  modelers  to  make  assumptions  about  the 
potential  impacts  of  these  technologies  in  modeling  projected  results.  While  the  previous 
work  has  provided  defensible  estimates  of  potential  improvements  (in  returns  on  investment, 
ROI)  and  cost  savings,  the  validity  and  usefulness  of  these  models  has  been  limited  by  the 
lack  of  comparative  data  on  ship  maintenance  processes  and  technology  investments,  and 
of  their  potential  impacts  on  performance.  Therefore,  the  acquisition  of  data  on  Dutch  naval 
fleet  maintenance  processes  and  the  comparison  of  those  data  with  previous  U.S.  Navy 
results  were  critical  steps  in  improving  U.S.  naval  technology  acquisition  decision-making,  in 
particular  with  regard  to  ship  maintenance. 
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To  be  valuable,  the  data  source  or  sources  for  this  work  had  to  have  several  critical 
similarities  with  U.S.  naval  ship  maintenance  processes.  The  data  source  had  to  consider 
technological  innovation  and  the  adoption  of  advanced  technologies  to  be  an  important  part 
of  its  naval  maintenance  acquisition  strategy.  The  data  source  or  sources  had  to  be  large 
enough  to  support  continuous  ship  maintenance  operations  because  the  intermittent 
stopping  and  restarting  of  operations  would  not  be  consistent  with  important  assumptions  of 
the  modeling  approach.  Finally,  the  data  source  had  to  be  accessible,  willing  to  share  the 
data,  and  willing  to  allow  us  to  obtain  the  new  data  required  for  our  modeling  approach. 
These  and  other  criteria  limited  the  potential  pool  of  sources  to  nations  or  large  industrial 
ship  maintenance  organizations  that  were  on  good  terms  with  the  United  States,  advanced 
enough  in  their  operations  to  compare  with  those  of  the  U.S.  Navy,  progressive  enough  in 
their  strategies  to  include  continuous  technology  adoption,  and  willing  to  share  data  and 
information  that  is  often  considered  essential  for  national  security  or  competitive  advantage. 
Damen  Industries  and  the  RDN  met  most  of  these  criteria  and  were  willing  to  meet  our 
requirements  for  data  acquisition  and  sharing. 

The  current  work  addresses  the  following  questions: 

•  How  are  the  Dutch  using  and  preparing  to  adopt  advanced  technologies, 
such  as  3D  LST  and  CPLM,  in  shipbuilding  and  maintenance? 

•  What  are  the  potential  changes  in  ROIs  provided  by  the  adoption  of  these 
advanced  technologies? 

•  How  do  those  potential  returns  compare  with  projected  estimates  of  returns 
on  technology  adoption  of  3D  LST  and  CPLM  in  the  U.S.  Navy? 

Research  Methodology  and  Background1 

The  traditional  ROI  equation  is  typically  expressed  as  (revenue  - 
investmentj/investment,  which  represents  the  productivity  ratio  of  output  (i.e.,  revenue  in 
ROI  -*■  input  or  investment  cost  in  ROI).  Accomplishing  this  analysis  in  a  nonprofit 
environment  presents  challenges  because  there  is  no  actual  revenue  generated.  Cost 
savings  from  reductions  in  manpower  requirements  (i.e.,  time  allocated  to  employee 
workload  for  various  tasks)  is  available  to  provide  the  impact  on  the  denominator  of  the  ship 
maintenance  efforts.  The  knowledge  value  added  (KVA)  methodology  (Housel  &  Kanevsky, 
1995)  also  allows  for  generation  of  a  quantifiable  surrogate  for  revenue  in  the  form  of 
common  units  of  output  described  in  terms  of  units  of  learning  time.  Specifically,  the  KVA 
methodology  allowed  the  study  team  to  quantify  the  knowledge  embedded  in  the  new 
processes  to  use  in  generating  common  units  of  output  estimates.  The  KVA  analysis 
provided  the  basic  ROI  estimates  critical  in  forecasting  the  future  value  of  various 
automation  options. 

The  system  dynamics  methodology  was  used  to  model  the  impacts  of  automation  on 
operations.  System  dynamics  applies  a  control  theory  perspective  to  the  design  and 
management  of  complex  human  systems.  System  dynamics  combines  servo-mechanism 
thinking  with  computer  simulation  to  create  insights  about  the  development  and  operation  of 
these  systems.  Forrester  (1961)  developed  the  methodology’s  philosophy,  and  Sterman 
(2000)  specified  the  modeling  process  with  examples  and  described  numerous  applications. 
System  dynamics  is  used  to  build  causal-based  (versus  correlation-based)  models  that 
reflect  the  components  and  interactions  that  drive  behavior  and  performance.  The 
methodology  has  been  used  extensively  to  explain,  design,  manage,  and,  thereby,  improve 


1  See  Ford,  Housel,  and  Mun  (2012)  for  a  more  detailed  description  of  the  research  methodologies 
applied. 
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the  performance  of  many  types  of  systems,  including  development  projects.  The 
methodology’s  ability  to  model  many  diverse  system  components  (e.g.,  work,  people, 
money,  value),  processes  (e.g.,  design,  technology  development,  production,  operations, 
quality  assurance),  and  managerial  decision-making  and  actions  (e.g.,  forecasting,  resource 
allocation)  makes  system  dynamics  useful  for  modeling  and  investigating  military  operations, 
the  design  of  materiel,  and  acquisition. 

The  integrated  risk  management  (IRM)  framework  and  supporting  toolset  was  used 
to  optimize  the  portfolio  over  time.  IRM  is  an  eight-step,  quantitative,  software-based 
modeling  approach  for  the  objective  quantification  of  risk  (cost,  schedule,  technical), 
flexibility,  strategy,  and  decision  analysis.  The  method  can  be  applied  to  program 
management,  resource  portfolio  allocation,  return  on  investment  to  the  military  (maximizing 
expected  military  value  and  objective  value  quantification  of  nonrevenue  government 
projects),  analysis  of  alternatives  or  strategic  flexibility  options,  capability  analysis,  prediction 
modeling,  and  general  decision  analytics.  The  method  and  toolset  provide  the  ability  to 
consider  hundreds  of  alternatives  with  budget  and  schedule  uncertainty  and  provide  ways  to 
help  the  decision-maker  maximize  capability  and  readiness  at  the  lowest  cost.  This 
methodology  is  particularly  amenable  to  resource  reallocation  and  has  been  taught  and 
applied  by  the  authors  for  the  past  10  years  at  over  100  multinational  corporations  and  over 
30  projects  at  the  U.S.  Department  of  Defense  (DoD). 

The  research  team  collected  data  on  Dutch  ship  operations  as  described  in  the 
section  titled  Data  Collection  Methods  and  used  it  to  build  three  types  of  computer 
simulation  models  of  ship  maintenance  and  technology  adoption:  KVA  models  of  return  on 
technology  investments  in  those  operations,  system  dynamics  models  (SD)  of  ship 
maintenance  operations,  and  IRM  models  of  implementation  plans  for  technology  adoption. 
The  results  were  then  analyzed  and  compared  with  previously  developed  modeling  results 
of  U.S.  Navy  ship  maintenance  and  technology  adoption. 

Data  Collection  Methods 

Data  on  the  practices  of  Dutch  industry  and  naval  ship  maintenance  proved  very 
difficult  and  time  consuming  to  obtain.  Initial  contact  with  Dutch  industry  participants  and 
ship  maintenance  technology  providers  developed  slowly  over  several  months  into 
relationships  that  eventually  led  to  data  collection  opportunities.  Several  sources  of  data 
were  utilized,  including  a  Dutch  shipbuilder  (Damen)  and  the  RDN.  Data  on  the  use  of 
technology  in  Dutch  fleet  maintenance  was  collected  by  two  primary  methods:  (1)  in-person 
interviews  and  meetings  with  managers  of  the  leading  corporation  in  the  Dutch  shipbuilding 
industry  (Damen)  and  with  officers  and  civilian  employees  of  the  RDN,  and  (2)  tours  of  three 
Dutch  shipbuilding  and  maintenance  facilities. 

In-person  interviews  and  meetings  with  managers  at  Damen  and  with  officers  and  a 
civilian  employee  of  the  RDN  occurred  during  a  data  collection  trip  by  one  of  the  research 
team  members  (Ford)  to  the  Netherlands  in  June  2012,  as  did  the  tours  of  Dutch  ship 
building  and  maintenance  facilities.  Meetings,  semi-structured  interviews,  and  extended 
discussions  were  held  with  six  managers  of  Damen  Industries  and  the  RDN  in  three 
locations  over  three  days.  At  these  meetings,  Damen  managers  made  presentations  on 
Damen’s  operations,  uses  of  technologies,  investigations  of  specific  technologies  for 
potential  development  and  adoption  (including  3D  LST  and  CPLM  software),  Integrated 
Logistics  System  (ILS),  and  information  technology  products  under  development  for  use  in 
ship  maintenance.2  Separately,  a  meeting  and  semi-structured  interview  was  conducted  with 


2  Copies  of  these  presentations  were  requested,  but  not  provided.  Data  collection  results  are  based 
on  notes  taken  by  the  investigator  during  the  meetings,  interviews,  and  tours  of  facilities. 
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the  two  RDN  officers  responsible  for  ship  maintenance  at  the  RDN  shipyard  at  Nieuwe 
Haven  in  Den  Helder.  Tours  of  the  RDN  fleet  maintenance  facility  in  Nieuwe  Haven  and  two 
Damen  shipyards  were  provided  during  the  data  collection  trip. 

Data  Collection  Results 

Damen’ s  Use  of  Technology 

The  Damen  Shipyards  Group  (www.damen.nl/)  is  a  large  Dutch  shipbuilding  firm  with 
worldwide  operations  (11  shipyards  with  five  outside  The  Netherlands).  The  firm  was  started 
in  1922  by  Jan  and  Rien  Damen.  The  firm  grew  substantially  after  Kommer  Damen  (the 
current  owner)  bought  it  in  1969  and  introduced  modular  and  standardized  shipbuilding  to 
the  industry.  The  firm  now  employs  over  6,000  people  and  builds  an  average  of  150  vessels 
per  year.  The  firm  obtained  Damen  Schelde  in  2000,  which  focuses  exclusively  on  naval 
ship  design,  building,  and  maintenance.  Damen  Schelde  manufactures  an  average  of  one  to 
two  ships  per  year,  employs  about  550  people,  and  performs  about  €210  million  per  year. 
Damen  Schelde  acts  as  the  prime  contractor  and  integrator  on  its  shipbuilding  projects, 
utilizing  many  subcontractors.  Although  Damen  Schelde  provides  ship  maintenance 
services  to  its  international  (i.e.,  not  Dutch)  customers,  it  does  not  provide  any  ship 
maintenance  services  for  the  RDN. 

Damen  Schelde  has  used  an  ILS  since  2002  to  manage  the  shipbuilding  process 
from  project  initiation  through  the  development  of  a  logistics  plan  for  customers.  The  ILS  is 
the  plan  for  the  development  of  a  ship  and  includes  ship  design;  production;  quality 
assurance,  quality  control  (QAQC);  training  of  ship  operators;  and  coordination  with 
customers.  The  ILS  does  not  include  service  contracts  or  life-cycle  costs  due  to  the  difficulty 
of  forecasting  those  costs.  The  focus  of  the  ILS  is  to  provide  maximum  ship  operational 
availability,  reliability,  and  maintainability.  It  does  this  partially  by  using  a  single  point  of 
contact  within  Damen  throughout  the  project  who  manages  an  interdisciplinary  team  (e.g., 
engineering,  work  preparation,  procurement,  service).  Damen  Schelde  currently  uses  a 
variety  of  information  technologies  to  facilitate  their  ILS  approach  to  shipbuilding  and  is 
constantly  investigating  new  technologies  that  may  improve  its  design  and  manufacturing. 

Of  particular  relevance  to  the  current  work,  Damen  Schelde  uses  four  separate  software 
products  to  manage  its  shipbuilding:  an  advanced  three-dimensional  CADD  program  for 
design,  a  CPLM  product  as  a  database  for  ship  components,  an  Enterprise  Resource 
Planning  (ERP)  system,  and  a  software  tool  for  scheduling.  The  latter  three  of  these 
systems  are  connected  to  users  with  a  project  information  portal  developed  by  Damen 
Schelde.  The  informant  reported  that  Damen  developed  the  portal  because  the  CPLM 
product  did  not  include  adequate  user  interfaces. 

Damen  Schelde  has  investigated  and  is  currently  investigating  other  technologies  for 
potential  adoption.  Four  technologies  were  described  and  discussed: 

1 .  3D  LST :  This  technology  was  investigated  but  was  assessed  to  currently  be 
too  immature  for  adoption  by  Damen  Schelde.  The  investigation  included  a 
discussion  of  the  current  use  of  the  technology  in  the  automobile  industry,  as 
well  as  its  potential  use  to  scan  engine  rooms  and  for  floor  flattening.  The  use 
of  360-degree  photography  (often  used  in  conjunction  with  3D  LST)  was 
considered  by  Damen  Schelde  as  a  potential  tool  for  training  (see  Komoroski, 
2005,  for  more  details  on  3D  LST). 

2.  3D  PDF  files:  Three-dimensional  animated  “movies”  of  shipbuilding  can  be 
created  in  a  PDF  format  (by  Adobe  Acrobat®)  and  sent  to  shipyards  for  use 
in  the  field  by  craftsmen  who  view  the  file  on  an  electronic  reader  (e.g.,  an 
iPad®).  The  files  would  replace  flat  drawings  for  use  in  construction.  The  file 
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visually  communicates  the  sequence  of  building  (or  maintenance)  operations 
and  components,  and  operations  can  have  notes  attached  to  them  that 
provide  additional  information  (e.g.,  part  numbers  or  warnings  of  special 
issues).  The  ability  to  animate  these  files  allows  engineers  to  visually  show 
craftsmen  sequences  of  operations,  routes  of  access  and  egress  for  line 
replaceable  units  (LRU3),  and  other  information  that  is  difficult  or  impossible 
to  show  with  traditional,  static,  two-dimensional  drawings.  The  use  of  this 
technology  shifts  the  understanding  of  the  design  intention  from  the  designers 
(in  the  Netherlands)  to  the  shipbuilding  yard  (typically  in  other  countries 
around  the  world).  The  use  of  visual  information  (the  animation  of  steps)  is 
expected  to  greatly  improve  communication  across  languages  since  many  of 
the  craftsmen  in  Damen’s  shipyards  do  not  read  English  well.  Damen 
considers  improvements  in  information  content  communicated  to  be  the 
primary  benefit  of  this  system  (versus  cost  savings).  Damen  Schelde  is  very 
optimistic  about  the  potential  for  this  technology  to  improve  its  operations  and 
is  actively  working  on  developing  it  (e.g.,  selecting  software,  addressing  the 
importing  of  the  3D  design  drawings).  Generating  the  animated  files  and 
adding  the  building  steps  to  the  design  files  is  expected  to  be  relatively  easy 
once  the  system  has  been  developed. 

3.  SIGMA  Shipbuilding  Strategy:  This  is  a  standardized  process  for  creating  a 
ship  that  spans  from  design  through  materials  procurement,  production,  and 
testing  of  a  ship.  The  key  feature  of  the  strategy  is  the  use  of  modular  ship 
sizes  and  systems  that  can  be  easily  adapted  to  specific  customer  needs.  For 
example,  Damen  Schelde  has  disaggregated  an  entire  ship  into  five 
standardized  modules  (e.g.,  fore,  midship,  aft)  with  major  systems  located  in 
specific  sections.  Each  module  is  considered  a  subproject.  As  an  example  of 
an  advantage  provided  by  the  strategy,  the  modules  and  their  interfaces  are 
designed  such  that  the  ship  can  be  made  longer  by  adding  an  additional 
midsection.4 

4.  Radio  Frequency  Identification  (RFID):  This  established  technology  is  being 
considered  for  use  to  improve  Damen’s  supply  chain  management.  Primary 
benefits  are  believed  to  be  improved  value  of  information  and  a  reduction  in 
the  duration  for  getting  information  into  Damen  databases  (e.g.,  warehouse 
contents,  components  on  specific  ships).  Both  passive  and  active  tags  are 
being  considered. 

Damen  Services  also  develops  advanced  technologies  for  use  by  Damen 
Enterprises.  Damen  Services  focuses  on  providing  ongoing  maintenance  parts  and  services 
to  Damen  customers  after  a  ship  has  been  designed,  built,  and  delivered,  but  also  provides 
other  services  such  as  civil  works  (e.g.,  wharves  and  storage  facilities). 

The  Maintenance  and  Spares  department  maintains  information  on  ship 
configuration  (using  an  ERP  system),  parts  inventories,  spare  parts  packages,  and 
maintenance  management  systems.  It  also  provides  information  technology  support  for 
Damen.  It  is  developing  a  web  portal  for  clients  that  will  allow  clients  access  to  Damen-held 
data  on  each  of  the  customer’s  ships  down  to  the  individual  component  level.  This  will 


3  Line  replaceable  unit  is  a  commonly  used  term  in  manufactured  devices  for  any  modular  component 
that  is  designed  to  be  interchangeable. 

4  This  portion  of  the  SIGMA  strategy  applies  the  Boeing  strategy  for  the  design  and  production  of  the 
737  that  has  different  lengths  to  shipbuilding. 
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partially  be  accomplished  with  a  work  breakdown  system  (WBS)  that  disaggregates  a  ship 
or  system  into  product  parts  (e.g.,  engine,  bilge  pump)  and  a  functional  breakdown  system 
that  disaggregates  the  ship  into  functions  (e.g.,  port  propulsion)  that  are  met  with  a  product 
part  (in  the  WBS)  and  have  an  associated  maintenance  schedule,  which  includes  monitoring 
measurements  and  frequency,  parts  documentation,  and  so  forth.  The  WBS  has  three 
levels:  subsystems  (e.g.,  propulsion,  hoisting),  with  a  typical  ship  having  20-70  subsystems; 
Level  2  parts  (e.g.,  pump,  shaft),  with  about  1,000  per  ship;  and  Level  3  parts  (e.g.,  bolt, 
flange),  with  70,000-80,000  per  ship. 

This  system  will  be  linked  with  an  online  parts  ordering  portal  so  that  customers  can 
order  parts  from  Damen  (similar  to  Amazon’s  online  selling  of  books,  etc.).  Damen  Services 
plans  to  use  the  information  (e.g.,  the  frequency  orders  for  specific  components)  captured 
through  this  system  to  develop  maintenance  optimization  information.  Damen  Services 
envisions  three  types  of  maintenance:  corrective  maintenance  (after  the  component  needs 
work),  preventative  maintenance  (based  on  forecasts  of  maintenance  needs),  and  condition- 
based  maintenance  (based  on  actual  conditions  of  components).  Condition-based 
maintenance  is  an  optimized  version  of  preventative-based  maintenance  that  is  currently 
under  development.  It  requires  sensors  to  collect  data  on  component  conditions  that  will  be 
used  to  generate  condition  assessments. 

Royal  Dutch  Navy  Fleet  Maintenance 

Data  collection  directly  from  the  RDN  was  particularly  valuable  for  at  least  two 
reasons.  First,  as  the  navy  of  a  sovereign  country  with  objectives  that  are  similar  to  those  of 
the  United  States,  the  objectives  and  issues  of  the  RDN  are  more  likely  to  match  those  of 
the  U.S.  Navy  than  those  of  some  other  nations.  Data  collection  supported  this  assumption. 
For  example,  technology  leadership,  interoperability,  and  reliability  in  meeting  operational 
needs  are  paramount  to  the  RDN,  and  the  RDN  has  recently  experienced,  and  expects  to 
continue  to  experience,  reductions  in  budgets  just  as  is  the  case  with  the  U.S.  Navy.  The 
Dutch  navy  continues  to  face  budget  cuts  and  increasing  technology  needs,  is  currently  in 
reorganization  to  reduce  total  workforce  (internal  to  the  navy  and  civilian  naval  workforce)  by 
20%,  and  is  transferring  from  legacy  information  systems  to  an  integrated  ERP  system  for 
maintenance  operations.  Also,  the  RDN  performs  all  of  the  maintenance  on  its  fleet,  thereby 
making  it  the  primary  data  source  concerning  RDN  fleet  maintenance  process  performance. 

The  interviews  with  the  two  RDN  officers  in  the  Naval  Maintenance  and  Service 
Agency  provided  a  general  introduction  to  the  issues  faced  by  the  Dutch  navy  in  building 
and  maintaining  its  fleet.  The  RDN  addresses  its  challenges  by  means  similar  to  those  used 
by  the  U.S.  Navy,  such  as  waiting  for  technology  to  mature  (technology  readiness  level 
[TRL]  >  7  before  adoption)  and  incremental  capability  increases  based  on  budgets. 
Noticeably  different,  both  the  RDN  and  Damen  described  the  critical  role,  and  standard 
Dutch  practice,  of  adjusting  requirements  to  meet  budgets  in  shipbuilding.  The  RDN  is 
facing  increasing  pressure  to  control  life-cycle  costs  in  its  fleet,  which  are  largely  driven  by 
personnel  and  fuel.  This  has  led  it  to  approve  significantly  stricter  operations  manning 
requirements  for  ship  design  (i.e.,  lower  maximum  shipboard  personnel),  which  has  driven 
Damen  to  increase  the  use  of  automation  in  its  ship  designs. 

The  primary  informant  on  RDN  fleet  maintenance  operations  provided  a  diagram  of 
those  operations  (see  Figure  1)  and  a  written  description  of  each  of  the  steps  identified  in 
the  diagram. 
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Logistics  Process  Royal  Netherlands  Navy 


Figure  1.  Diagram  of  Royal  Dutch  Navy  Fleet  Maintenance  Processes 

(P.  Kense,  personal  communication,  June  21, 2012) 

The  process  steps  shown  in  Figure  1  were  described  in  writing  by  the  informant  with 
the  following  list.5  In  the  list,  the  abbreviation  LRU  stands  for  line  replaceable  unit,  a 
commonly  used  term  in  the  area  of  manufactured  devices  for  any  modular  component  that  is 
designed  to  be  interchangeable. 

Logistic  Process  Royal  Netherlands  Navy 

1.  In  case  LRU  fails  the  on-board  personnel  will  replace  this  LRU  by  a  spare 
(on-board;  OLM  qualification  required). 

2.  The  defect  LRU  will  be  send  to  the  warehouse,  and  a  “new”  LRU  will  be  send 
to  the  ship. 

3.  The  defect  LRU  will  be  send  to  the  Naval  Maintenance  Establishment  (NME) 
for  repair.  After  the  LRU  is  repaired  it  will  be  send  to  the  warehouse  again  “as 
good  as  new”  (DLM  qualification  required). 

4.  If  the  NME  needs  parts  to  repair  an  LRU,  the  parts  will  be  extracted  from  the 
industry,  when  the  NME  is  not  able  to  repair  this  LRU,  it  can  be  send  to  the 
manufacturer.  Also,  manpower  can  be  hired  to  fix  problems. 

5.  If  spare  is  not  available,  sometimes  it  will  be  cannibalized  from  another  ship. 

6.  If  the  on-board  personnel  is  not  able  to  fix  the  problem  by  themselves  (due  to 
the  complexity  of  the  failure)  assistance  from  the  NME  is  needed  (ILM 
qualification  required). 

7.  If  the  problem  is  too  complex  for  the  NME  also,  the  industry  can  be  hired  to 
solve  this  problem. 


5  The  process  step  descriptions  have  been  transcribed  exactly  as  provided  in  English  by  the  RDN, 
including  uncommon  English  grammar  and  spelling. 
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The  following  seven  process  steps  were  elaborated  on  by  the  informant  (the 
abbreviations  DLM ,  OLM,  and  ILM  refer  to  Dutch  terms  for  training  levels): 

Step  1:  Performed  onboard,  for  example  to  provide  operational 
maintenance  of  weapons  systems 

Step  2:  Purely  a  transit  operation  that  requires  only  a  truck  driver  (if  ship  is 
in  port) 

Step  4:  Requires  DLM  level  of  training 

Step  5:  Requires  OLM  level  of  training 

Step  6:  Requires  ILM  level  of  training  (=  LTS  +  MTS  +  10-25  days  of 
training) 

Step  7:  Requires  DLM  level  of  training 

Fleet  maintenance  for  the  RDN  requires,  at  a  minimum,  completion  of  education  at  a 
Lower  Technical  School  (LTS)  and  a  Middle/Intermediate  Technical  School  (MTS).  The  LTS 
is  typically  attended  between  ages  12-16,  and  the  MTS  is  typically  attended  between  ages 
1 6-21 .  After  completion  of  LTS  and  MTS,  future  RDN  ship  fleet  maintenance  personnel 
must  complete  at  least  one  of  three  other  forms  of  training. 

System  Dynamics  Model  Structure 

The  system  dynamics  model  simulates  the  movement  of  LRU  among  the  various 
locations  where  they  are  used,  stored,  or  repaired.  Each  flow  of  LRUs  between  two  stocks 
represents  the  processing  rate  of  one  of  the  process  steps  in  a  KVA  model.  A  simplified 
diagram  of  the  stocks  and  flows  of  the  model  are  shown  in  Figure  2.  Boxes  represent 
stocks,  or  accumulations  of  LRU.  Each  stock  in  Figure  2  represents  a  location  in  Figure  1 , 
plus  on-board  LRU  storage  as  a  separate  LRU  accumulation.  Arrows  with  valve  symbols  in 
Figure  2  represent  the  movement  of  LRUs  between  stocks.  Numbers  in  parentheses  in  the 
titles  of  flows  represent  the  process  steps  shown  in  Figure  1  (ovals  with  arrows)  and  the 
KVA  model  process  steps  (described  later). 
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Figure  2.  Royal  Dutch  Navy  Ship  Maintenance:  Stocks  and  Flows  of  the  System 

Dynamics  Model 

The  sizes  of  the  flows  in  the  system  dynamics  model  describe  the  rate  of  movement 
of  LRUs  among  the  stocks.  Therefore,  the  simulated  flows  in  the  system  dynamics  model 
become  direct  inputs  to  the  “times  processed  per  year”  portion  of  the  KVA  models.  Flow 
rates  were  modeled  to  reflect  the  sequence  of  processes  in  operations.  For  example,  in 
normal  operations,  the  replacement  of  a  broken  LRU  in  an  operating  ship  with  one  from  the 
ship’s  on-board  storage  (“Replace  broken  LRU  from  storage  [1]”  on  the  left  of  Figure  2) 
would  be  followed  by  the  broken  LRU  in  storage  being  replaced  by  an  operational  LRU  from 
the  warehouse  (“Replace  broken  LRU  from  warehouse  on  onboard  storage  [8]”  at  the  top  in 
Figure  2).  This  replacement  would  be  followed  by  the  broken  LRU  being  sent  to  the  NME 
where  it  would  be  repaired  and  returned  to  the  warehouse  (“NME  repairs  broken  LRU  in 
warehouse  [3]”  on  the  right  in  Figure  2).  These  precedencies  are  modeled  by  having  the 
downstream  process  equal  to  its  preceding  process  step  with  a  delay  that  reflects  the  transit 
and  subsequent  processing  time.  Some  flows  (e.g.,  “NME  repairs  broken  LRU  from 
warehouse  [3]”)  are  aggregations  of  multiple  upstream  flows.  Core  flows  are  based  on  the 
mean  time  between  failure  of  LRUs  and  the  fraction  of  failures  addressed  with  each 
process. 

The  system  dynamics  model  was  calibrated  to  reflect  RDN  ship  maintenance  (see 
Ford,  Housel,  &  Mun,  2012,  for  details). 

Knowledge  Value  Added  Models  to  the  Royal  Dutch  Navy  Ship  Maintenance 

Four  KVA  models  were  built  based  on  the  RDN  ship  maintenance  processes  (see 
Ford,  Housel,  &  Dillard,  2010,  for  details  and  examples  of  KVA  modeling): 

1.  Baseline  RDN  ship  maintenance  processes 
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2.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the  adoption 
and  use  of  a  logistics  package  from  an  integrated  CPLM  system  such  as  was 
investigated  by  Damen 

3.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the  adoption 
and  use  of  3D  PDF  modeling  managed  with  a  CPLM  system  as  planned  by 
Damen 

4.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the  adoption 
and  use  of  a  logistics  package  and  3D  PDF  modeling  managed  by  an 
integrated  CPLM  system 

Model  Simulations  and  Results 

The  system  dynamics  model  was  simulated  to  represent  the  four  technology 
adoption  scenarios  described  in  the  previous  section.  The  output  of  each  system  dynamics 
model  simulation  was  used  as  input  to  a  KVA  model.  Those  KVA  models  were  then  used  to 
estimate  the  ROI  of  each  process  in  each  of  the  four  scenarios  and  the  cumulative  ROI  for 
each  scenario.  The  results  based  on  the  models  and  their  calibrations  are  shown  in  Table  1. 


Table  1.  Knowledge  Value  Added  Model  Results 


Return  On  Investment  (ROI) 

Process 

Description 

Baseline 

Add 

Logistics 

Add  3D 

PDF 

Add 

Logistics 
&  3D  PDF 

1 

Replace  LRU  with  on-board 
spare 

90% 

261% 

501% 

464% 

2 

Replace  operating  LRU  with 
warehouse  spare 

90% 

151% 

621% 

1 027% 

3 

NME  repairs  warehouse  LRU 
and  returns  it  to  warehouse 

8% 

65% 

95% 

236% 

4 

Manufacturer  repairs  LRU  for 
NME  &  it  returns  to  warehouse 

31% 

88% 

168% 

1 68% 

5 

Replace  on-board  LRU  with 

LRU  cannibalized  from  another 
ship 

90% 

151% 

621% 

1 027% 

6 

NME  repairs  on-board  LRU  and 
returns  it  to  ship 

265% 

10% 

99% 

192% 

7 

Industry  repairs  on-board  LRU 
and  returns  it  to  ship 

34% 

1 78% 

1 35% 

318% 

8 

Replace  on-board  storage  LRU 
with  warehouse  spare  (transit 
only) 

301% 

759% 

759% 

759% 

9 

Replace  cannabalized  LRU  with 
warehouse  spare  (transit  only) 

140% 

329% 

862% 

1 1 02% 

TOTAL  ALL  PROCESSES 

35% 

77% 

1 35% 

274% 

Although  increased  throughput  due  to  reduced  processing  durations  (which  increase 
the  ROI  numerator)  can  partially  explain  differences  in  the  ROI  in  Table  1,  cost  reduction 
(which  decreases  the  ROI  denominator)  is  the  primary  driver  of  increases  in  ROI.  For 
example,  Processes  8  and  9  are  benefited  by  reductions  in  rework  (e.g.,  errors  in 
transporting  LRU)  due  to  the  adoption  of  a  logistics  package.  This  reduces  the  number  of 
transport  trips  required  (the  function  of  these  processes),  thereby  significantly  reducing 
costs  and  increasing  the  ROI.  In  contrast,  Processes  3,  4,  and  6  are  highly  skilled  processes 
that  are  difficult  to  replace  with  technology  and,  therefore,  benefit  less  from  technology 
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adoption  than  other  processes.  This  results  in  a  smaller  increase  in  ROI  for  these 
processes. 

Analysis  of  Simulation  Model  Results 

A  variance  analysis  was  performed  on  the  KVA  model  results  (Table  1)  to  evaluate 
the  relative  impacts  of  the  adoption  of  different  technologies  (Table  2).  ROIs  for  each  of  the 
three  technology  adoption  alternatives  were  compared  with  the  baseline  ROIs  to  estimate 
improvement  due  to  technologies  (see  the  left  three  columns  of  results  in  Table  2).  In 
addition,  the  improvement  from  adopting  both  technologies  over  adopting  only  the  3D  PDF 
technology  was  estimated  (see  the  right  column  in  Table  2). 

Table  2.  Variance  Analysis  of  Knowledge  Value  Added  Model  Results 


Return  On  Investment  (ROI) 

Process 

Description 

Add  Logistics 
Improvement 
over  Baseline 

Add  3Dpdf  - 
Improvement 
over  Baseline 

Add  Logistics 
&  3Dpdf  - 
Improvement 
over  Baseline 

Add  Logistics  & 
3Dpdf  - 
Improvement 
over  adding  only 
3Dpdf 

1 

Replace  LRU  with  on-board 
spare 

171% 

411% 

374% 

-38% 

2 

Replace  operating  LRU  with 
warehouse  spare 

61% 

532% 

937% 

406% 

3 

NME  repairs  warehouse  LRU 
and  returns  it  to  warehouse 

57% 

87% 

227% 

140% 

4 

Manufacturer  repairs  LRU  for 
NME  &  it  returns  to  warehouse 

57% 

138% 

138% 

0% 

5 

Replace  on-board  LRU  with 

LRU  cannibalized  from  another 
ship 

61% 

532% 

937% 

406% 

6 

NME  repairs  on-board  LRU  and 
returns  it  to  ship 

-256% 

-166% 

-73% 

93% 

7 

Industry  repairs  on-board  LRU 
and  returns  it  to  ship 

145% 

101% 

284% 

183% 

8 

Replace  on-board  storage  LRU 
with  warehouse  spare  (transit 
only) 

458% 

458% 

458% 

0% 

9 

Replace  cannabalized  LRU  with 
warehouse  spare  (transit  only) 

189% 

721% 

962% 

240% 

TOTAL  ALL  PROCESSES 

42% 

100% 

239% 

139% 

Referring  to  Table  2,  adding  either  or  both  of  the  technologies  improves  overall  ship 
maintenance  ROI,  as  indicated  by  the  positive  numbers  in  the  last  row  of  Table  2.  Adopting 
3D  PDF  alone  improves  ROI  significantly  more  than  adopting  a  logistics  package  alone 
(100%  improvement  >  46%  improvement),  and  adding  both  technologies  improves  ROI 
more  than  adding  either  technology  alone  (239%  improvement  >  42%  improvement  or  100% 
improvement),  suggesting  that  there  may  be  synergy  between  the  technologies.  This  is  also 
supported  by  the  139%  improvement  by  adding  logistics  if  3D  PDF  is  already  in  place  (see 
the  lower  right  result  in  Table  2). 

Adopting  the  technologies  does  not  impact  the  ROI  of  individual  processes  equally. 
Among  the  seven  core  processes  (1-7),  adding  only  a  logistics  package  (see  the  left  column 
of  results  in  Table  2)  increases  the  “Replace  LRU  with  on-board  spare”  (Process  1)  most,  by 
171%,  and  decreases  the  return  of  Process  6,  “NME  repairs  on-board  LRU  and  returns  it  to 
ship,”  by  256%.  Among  the  seven  core  processes,  adding  only  3D  PDF  increases 
Processes  2  and  5,  “Replace  operating  LRU  with  warehouse  spare”  and  “Replace  on-board 
LRU  with  LRU  cannibalized  from  another  shop”  most,  by  532%,  and  decreases  the  return  of 
Process  6,  “NME  repairs  on-board  LRU  and  returns  it  to  ship”  by  166%.  Among  the  seven 
core  processes,  adding  both  technologies  increases  Processes  2  and  5,  “Replace  operating 
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LRU  with  warehouse  spare”  and  “Replace  on-board  LRU  with  LRU  cannibalized  from 
another  shop”  most,  by  937%,  and  decreases  the  return  of  Process  6,  “NME  repairs  on¬ 
board  LRU  and  returns  it  to  ship,”  by  73%. 

Comparison  of  Royal  Dutch  Navy  and  U.S.  Navy  Scenarios 

Previous  research  using  the  KVA  approach  developed  estimates  of  returns  on 
technology  investment  of  a  scenario  in  which  the  U.S.  Navy  adopts  3D  LST  and  CPLM  tools 
into  the  SHIPMAIN  program.  Komoroski  (2005)  investigated  the  early  phases  of  SHIPMAIN 
(see  Table  3).  Adding  the  3D  LST  and  CPLM  technologies  improves  the  overall  preparation 
for  maintenance  process  ROI.  Adding  these  technologies  generally  improves  individual 
processes  as  well.  The  range  of  improvements  across  individual  processes  is  large,  varying 
from  0%  (Issue  Tasking)  to  3,031%  (Generate  Drawings).  Cost  reduction  explains  these 
differences.  For  example,  the  adoption  of  technology  in  Core  Processes  4  (Conduct 
Shipcheck)  and  7  (Generate  Drawings)  would  significantly  reduce  the  number  of  people 
required  to  survey  ship  conditions  (4)  or  draft  3D  drawings  from  the  survey  data  (9),  resulting 
in  large  ROI  if  the  technology  is  adopted. 

Seaman,  Housel,  and  Mun  (2007)  used  KVA  to  model  the  later  phases  of  SHIPMAIN 
(see  Table  3).  Adding  the  technologies  also  improves  overall  maintenance  implementation 
process  ROI.  Adding  these  technologies  also  improves  each  of  the  individual  processes. 

The  range  of  improvements  across  individual  processes  is  large,  varying  from  6%  to  466% 
(Final  Install,  Closeout  SC),  although  not  as  wide  as  in  the  preparation  for  maintenance 
processes  (see  Seaman,  Housel,  &  Mun,  2007,  for  details). 

Table  3.  Return  on  Investment:  Baseline  and  Technology  Adoption  Scenarios 


Baseline 

Overall  ROI 

Technology- 
adopted 
Overall  ROI 

US  Navy  -  SHIPMAIN 

(preparation  for 
maintenance  phases) 

-27% 

2019% 

US  Navy  -  SHIPMAIN 

(implementation  phases) 

35% 

201% 

Royal  Dutch  Navy 

(Damen  experience 
extrapolation) 

35% 

274% 

The  three  scenarios  have  some  similarities.  For  all  three,  overall  ROIs  after 
technology  adoption  are  positive  and  large.  This  supports  the  adoption  of  advanced 
technologies,  such  as  3D  LST,  3D  PDF  models,  and  CPLM,  to  improve  the  efficiency  of 
resource  use.  The  scenarios  also  have  potentially  significant  differences.  The  technology 
adoption  scenario  for  the  preparation  for  maintenance  phases  of  the  U.S.  scenario  has  a 
much  higher  overall  ROI  than  the  ROIs  for  the  maintenance  implementation  phases  of  the 
U.S.  or  the  Dutch  scenario  (2,019%  »  201%  or  274%).  Several  factors  could  explain  these 
differences. 

•  The  preparation  for  maintenance  phases  of  the  U.S.  scenario  have 
significantly  lower  ROI  in  the  As-ls  (without  technology)  condition  (-27%  > 
35%).  This  suggests  that  inefficiencies  in  the  preparation  for  maintenance 
processes  provided  more  and  larger  opportunities  for  improvement. 

•  The  individual  preparation  for  maintenance  processes  that  increased  the 
most,  such  as  Generate  Drawings  and  Conduct  Shipcheck,  are  very  labor 
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intensive  and,  therefore,  costly,  providing  large  opportunities  for  cost 
reduction  through  technology  adoption. 

•  Several  of  the  individual  maintenance  implementation  processes  are  labor 
intensive  but  less  impacted  by  technology  (e.g.,  Install  Shipcheck),  thereby 
making  those  changes  in  ROI  less  dramatic. 

•  The  preparation  for  maintenance  phases  of  the  U.S.  scenario  could  be  more 
optimistic  in  their  projections  than  the  other  scenarios. 

•  The  estimates  of  process  changes  may  use  different  assumptions. 

•  Technologies  adopted  in  the  preparation  for  maintenance  phases  of  the  U.S. 
scenario  may  make  much  larger  improvements  in  processes  than  those  in  the 
maintenance  implementation  phases  of  the  U.S.  or  the  Dutch  scenario. 

•  The  Dutch  case  does  not  use  all  of  the  capabilities  of  the  CPLM,  thereby 
making  it  more  incremental  than  the  U.S.  scenarios,  in  which  all  the 
capabilities  of  the  CPLM  were  projected  to  be  used.  Also,  3D  PDF  has  more 
limited  capabilities  for  integration  with  the  CPLM  logistics  package  when 
compared  to  the  integration  of  3D  LST  capabilities  for  broader  usage  in 
requirements  analysis,  planning  for  maintenance,  and  tracking  of  parts  in  the 
supply  chain  and  across  suppliers  and  contractors.  This  can  partially  explain 
the  lower  ROI  for  the  Dutch  technology-adopted  scenario  than  for  the  U.S. 
preparation  for  maintenance  scenario. 

•  The  projections  of  the  impacts  on  the  maintenance  implementation  phases  of 
the  U.S.  scenario  and  the  Dutch  scenario  may  be  rather  conservative  based 
on  research  into  the  actual  successful  implementation  of  other  modern 
technologies,  such  as  RFID  in  inventory  management.  In  a  study  of  the  actual 
use  of  passive  RFID  in  two  military  warehouses  in  the  Korean  air  force  and 
army,  the  actual  ROIs  from  use  of  the  RFID  technology  were  more  than  triple 
the  projected  impact  of  the  use  of  the  technology  in  a  separate  study  of  the 
U.S.  Navy  (Courtney,  1997).  The  Korean  ROIs  after  actual  implementation  of 
the  RFID  technology  ranged  from  610%  to  576%,  compared  to  the  projected 
returns  anticipated  from  the  implementation  of  the  same  technology  in  the 
U.S.  Navy,  which  ranged  up  to  133%.  The  implication  is  that  actual 
successful  implementation  of  information  technology  in  a  military  may  exceed 
projections  of  the  potential  impacts  of  the  technology.  It  follows  that  the 
current  research  on  the  impacts  of  CPLM  and  3D  LST  or  3D  PDF  may  be 
more  conservative  than  the  reality  once  these  technologies  are  actually 
implemented  on  a  wide-scale  basis. 

Integrated  Risk  Management  Modeling  and  Results 

Through  the  use  of  Monte  Carlo  simulation,  the  resulting  stochastic  KVA  ROK  model 
yielded  a  distribution  of  values  rather  than  a  point  solution.  Thus,  simulation  models  analyze 
and  quantify  the  various  risks  and  uncertainties  of  each  program.  The  result  is  a  distribution 
of  the  ROKs  and  a  representation  of  the  project’s  volatility. 

It  is  important  to  understand  why  it  is  necessary  to  apply  uncertainty  to  the  model. 
Because  the  KVA  process  provided  a  point  value  for  each  quantity,  even  though  there  was 
some  uncertainty  in  the  estimates  provided  by  the  subject-matter  experts,  application  of  the 
appropriate  statistical  distributions  of  input  was  used  to  restore  the  real  world’s  uncertainty 
to  the  model.  Having  inputs  from  only  three  experts,  as  opposed  to  hundreds  of  estimates, 
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and  rather  than  using  these  three  discrete  inputs,  we  applied  the  lessons  learned  in  cost 
estimating  as  reflected  in  the  Air  Force  handbook  (U.S.  Air  Force,  2007)  as  a  good  starting 
point  for  representing  the  uncertainty  and  reflecting  it  in  the  simulations. 

Next,  using  the  developed  KVA  model,  risk  simulation  probabilistic  distributional  input 
parameters  were  inserted  into  the  three  main  variables:  percentage  automation,  time 
process  is  executed,  and  average  time  to  complete.  A  risk  simulation  of  10,000-1,000,000 
simulation  trials  was  run  to  obtain  the  results. 

At  this  point  in  the  analysis,  a  proxy  for  revenues  and  volatility  has  been  identified,  as 
well  as  the  numerators  and  denominators  for  the  ship  maintenance  program.  The  next  step 
is  to  define  or  frame  the  alternatives  and  approaches  to  implementing  3D  PDF  and  Logistics 
Team  Centers,  namely,  strategic  real  options.  The  questions  that  can  be  answered  include 
the  following:  What  are  the  options  involved?  How  should  these  new  processes  be  best 
implemented?  Which  decision  pathway  is  optimal?  and  How  much  is  the  program  worth  to 
the  DoD? 

Integrated  Risk  Management:  Framing  the  Real  Options 

As  part  of  the  first  round  of  analysis,  Figure  3  illustrates  some  of  the  potential 
implementation  paths  for  3D  PDF/Logistics  TC.  Clearly,  some  of  the  pathways  and  flexibility 
strategies  can  be  refined  and  updated  through  the  passage  of  time,  actions,  and  events. 

With  the  evolution  of  the  implementation,  valuable  information  is  obtained  to  help  in  further 
fine-tuning  the  implementation  and  decision  paths. 

For  the  preliminary  analysis,  the  following  options  were  identified,  subject  to 
modification: 

•  Option  A:  As-ls  Base  Case.  The  ROI  for  this  strategic  path  is  computed  using 
the  baseline  KVA  and  this  represents  the  current  RDN  ship  maintenance 
process  (i.e. ,  no  newly  added  technologies). 

•  Option  B:  Execute  and  implement  3D  PDF  and  Logistics  package 
immediately  across  all  RDN  ship  maintenance  processes.  That  is,  take  the 
risk  and  execute  on  a  larger  scale,  where  you  would  spend  the  initial 
investments  and  continuing  maintenance  expenses  required  and  take  on  the 
risks  of  any  potential  failure,  but  reap  the  rewards  of  the  new  processes’ 
savings  quickly  and  immediately.  The  analysis  is  represented  as  the  current 
RDN  process  altered  to  reflect  what  we  estimate  to  be  the  impacts  of 
adopting  both  a  Logistics  package  and  3D  PDF  models. 

•  Option  C:  This  represents  the  current  RDN  process  altered  to  reflect  what  we 
estimate  to  be  the  impacts  of  adopting  3D  PDF  models  and  managing  them 
in  a  Team  Center  or  similar  product.  This  technology  was  chosen  largely 
because  Damen  is  developing  and  pursuing  the  use  of  this  technology. 

•  Option  D:  This  implementation  pathway  represents  the  current  RDN  process 
altered  to  reflect  what  we  estimate  to  be  the  impacts  of  managing  using  a 
Logistics  module  in  a  Team  Center  or  similar  product.  This  technology  was 
chosen  partially  because  it  was  a  technology  that  Damen  considered,  but 
chose  not  to  purchase. 

•  Option  E:  Proof  of  Concept  (POC)  approach.  That  is,  to  execute  large-scale 
implementation  of  3D  PDF  and  Logistics  Module  in  TC  only  after  an  initial 
POC  shows  promising  results.  If  POC  turns  out  to  be  a  failure,  we  walk  away 
and  exit  the  program,  and  losses  are  minimized  and  limited  to  the  initial  POC 
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expenses.  Proceed  to  full  implementation  in  POC  programs  first  and  then 
expand  in  sequential  fashion  to  other  programs,  based  on  where  best  ROI 
estimates  are  shown. 

•  Option  F:  POC  on  3D  PDF  only.  Assuming  the  POC  works  and  3D  PDF  is 
executed  within  a  few  programs  successfully,  the  learning  and  experience 
obtained  becomes  valuable  and  allows  the  shipyard  to  expand  its  use  into 
many  other  programs  or  perhaps  across  the  RDN. 

•  Option  G:  POC  on  Logistics  Module  in  TC  only.  Assuming  the  POC  works 
and  Logistics  Module  is  executed  within  a  few  programs  successfully,  the 
learning  and  experience  obtained  becomes  valuable  and  allows  the  shipyard 
to  expand  its  use  into  many  other  programs  or  perhaps  across  the  RDN. 
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Integrated  Risk  Management:  Strategic  Flexibility  Real  Options  Results 

Figure  4  shows  the  results  of  the  strategic  real  options  flexibility  values  and 
compares  them  against  the  KVA  ROI  values.  Options  B  ($154.1  million  at  278%  ROI)  and  E 
($156.5  million  at  282%  ROI)  of  implementing  both  3D  PDF  and  Logistics  Module  TC  return 
the  highest  ROI  and  total  strategic  value,  and  both  provide  a  significant  value-add  above 
and  beyond  Option  A’s  As-ls  condition  ($31.9  million  at  35%  ROI).  As  Options  B  and  E  are 
the  most  significant;  stage-gating  the  implementation  over  several  phases  yields  a  slightly 
higher  value  (Option  E  exceeds  Option  B  by  about  $2.4  million). 

In  addition,  the  Monte  Carlo  risk  simulation  results  on  the  real  options  values  were 
developed  (but  are  not  shown  here  for  brevity;  see  Ford,  Housel,  &  Mun,  2012).  In 
comparing  Options  E  and  F,  there  is  a  94%  probability  that  Option  E,  which  has  a 
sequentially  phased  implementation  of  both  3D  PDF  and  Logistics  Module  TC,  provides  a 
better  return  than  Option  F.  In  comparing  Options  E  and  B,  there  is  a  95%  confidence  that, 
even  with  all  the  uncertainties  in  the  collected  data  and  risks  of  implementation  success, 
including  uncertainties  of  whether  the  estimated  returns  will  materialize  and  so  forth,  there  is 
at  least  a  $1.27  million  net  advantage  in  going  with  Option  E.  Therefore,  it  is  better  to 
sequentially  phase  and  stage-gate  the  implementation  over  several  years,  allowing  the 
ability  to  exit  and  abandon  further  stages  if  events  unfold  and  uncertainties  become 
resolved,  so  that  further  investment  in  the  technology  no  longer  makes  sense.  The  risk- 
simulated  real  options  value  has  an  expected  value  (mean)  of  $195  million,  with  a 
corresponding  average  ROI  of  363%. 
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Figure  4.  Sample  Real  Options  Values 
Summary  Results  of  the  Integrated  Risk  Management  Analysis 

IRM  and  strategic  real  options  methodologies  were  applied  to  the  KVA-SD  results, 
and  the  results  indicate  that  Option  B  had  a  value  of  $154.1  million  (278%  ROI)  and  Option 
E  had  a  value  of  $156.5  million  (282%  ROI),  where  both  options  indicate  that  implementing 
3D  PDF  and  Logistics  Module  TC  return  the  highest  ROI  and  total  strategic  value  and  both 
provide  a  significant  value-add  above  and  beyond  Option  A’s  As-ls  condition,  with  a  value  of 
$31 .9  million  (35%  ROI).  As  Options  B  and  E  are  most  significant,  we  know  that 
implementation  of  3D  PDF  and  Logistics  Module  TC  returns  the  highest  value  and,  when 
implemented  over  time  in  a  stage-gate  process  over  several  phases,  would  yield  a  slightly 
higher  value  (Option  E  exceeds  Option  B  by  about  $2.4  million).  Therefore,  we  conclude  that 
3D  PDF  and  Logistics  Module  TC  implemented  in  a  phased  stage-gate  environment  would 
yield  the  best  results.  In  comparing  Options  E  and  B,  there  is  a  95%  probability,  even  with  all 
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the  uncertainties  in  the  collected  data  and  risks  of  implementation  success  as  well  as  the 
uncertainties  of  whether  the  estimated  returns  will  materialize,  that  there  is  a  $1 .27  million 
net  advantage  in  going  with  Option  E  to  sequentially  phase  and  stage-gate  the 
implementation  over  several  years,  allowing  the  ability  to  exit  and  abandon  further  stages  if 
events  unfold  and  uncertainties  become  resolved  so  that  further  investment  in  the 
technology  no  longer  makes  sense. 

Conclusions 

We  collected  new  data  on  ship  maintenance  processes  and  the  use  and  adoption  of 
technologies  in  ship  maintenance  by  the  RDN  and  Damen  Shipbuilding.  The  data  were  used 
to  build  and  calibrate  a  system  dynamics  model  of  RDN  ship  maintenance.  Model 
simulations  of  four  technology  adoption  scenarios,  reflecting  the  use  of  two  available  or 
developing  technologies,  generated  estimates  of  maintenance  operations  behavior  that 
were  imported  into  KVA  models.  The  four  technology  adoption  scenarios  were  then  modeled 
in  the  KVA  models.  The  KVA  models  estimated  the  ROIs  for  individual  processes  and  ship 
maintenance  as  a  whole  for  each  scenario.  Results  were  analyzed  to  reveal  the  relative 
improvement  provided  by  individual,  and  combinations  of,  technologies. 

The  results  of  this  study,  in  combination  with  prior  studies,  make  it  evident  that  the 
technologies  under  review  will  make  large  contributions  to  cost  reductions  in  ship 
maintenance  processes.  These  conclusions  are  supported  by  the  comparative  analysis  of 
the  Dutch  experience  with  similar  supporting  technologies.  There  appears  to  be  no  empirical 
evidence  that  would  serve  as  an  impediment  to  adopting  the  technologies  in  the  near  term 
rather  than  the  longer  term.  We  recommend  an  immediate  adoption  of  the  3D  LST  and 
CPLM  technologies  to  support  ship  maintenance  processes. 

Implications  for  Acquisition  Practice 

The  current  research  has  significant  implications  for  acquisition  practice.  First,  the 
conclusions  support  multiple  previous  investigations  that  recommend  the  adoption  of 
available  information  technologies  to  reduce  the  costs  of  U.S.  Navy  ship  maintenance. 
Second,  multiple  significantly  different  technologies  (e.g.,  3D  LST,  3D  PDF,  logistics 
support)  can  improve  ship  maintenance  operations.  Third,  among  those  studied,  the 
expensive  information  technologies  were  found  to  benefit  high-cost  processes  the  most, 
such  as  where  labor  can  be  replaced  with  technology.  Doing  so  reduces  costs  and 
increases  production  rates  by  reducing  cycle  times.  This  implies  that  if  technology  adoption 
efforts  are  to  be  prioritized,  those  with  labor-intensive  processes  that  can  be  replaced  with 
technology  should  be  given  higher  priority.  The  real  options  analysis  of  implementation 
strategies  demonstrated  that  some  technologies  (3D  PDF  in  this  case)  can  dominate  the 
value  space  and  that  phased  implementation  adds  value  compared  to  one-step 
implementation.  The  results  of  the  current  work  recommend  a  careful  investigation  of 
available  technologies  and  how  they  improve  operations,  followed  by  a  phased  development 
and  implementation  of  the  adoption  of  the  chosen  technologies. 

Implications  for  Research 

The  results  of  the  three  KVA-based  studies  varied  significantly.  A  likely  cause  is  the 
difficulty  in  accurately  forecasting,  in  quantitative  terms,  the  impacts  of  new  technologies  on 
specific  processes.  The  use  of  data  and  information  from  organizations  that  are  actively 
developing  and  adopting  information  technologies  (Damen)  and  performing  operations 
similar  to  those  performed  by  the  U.S.  Navy  (RDN)  proved  to  be  very  valuable  in  improving 
the  models  (e.g.,  by  adding  the  3D  PDF  technology).  Therefore,  further  refinement  of  the 
models  should  include  actual  application  data,  such  as  a  study  of  actual  technology 
adoptions  by  the  U.S.  Navy. 
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